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[Document Name] Specification 

[Title of the Invention] IMAGE ENCODING METHOD, IMAGE 
DECODING METHOD, IMAGE ENCODING APPARATUS, IMAGE 
DECODING APPARATUS, IMAGE PROCESSING SYSTEM, 
IMAGE ENCODING PROGRAM, AND IMAGE DECODING 
PROGRAM 
[Claims] 

[Claim 1] An image encoding method of dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding thereof, the image 
encoding method comprising: 

a block selecting step of selecting a size of a block for the 
orthogonal transform, out of a plurality of blocks of different sizes; 

a coefficient string dividing step of, when a block of a size 
larger than a minimum size is selected in the block selecting step, 
dividing a coefficient string in said block into a plurality of coefficient 
strings of a length equal to that of a coefficient string in a block of the 
minimum size; and 

an encoding step of performing entropy coding adapted to the 
coefficient string in the block of the minimum size. 

[Claim 2] The image encoding method according to Claim 1, 
wherein the coefficient string dividing step is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and assign the read coefficients one by one in order to the 
plurality of coefficient strings of the length equal to that of the 
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coefficient string in the block of the minimum size, thereby obtaining 
the divided coefficient strings, 

[Claim 3] The image encoding method according to Claim 1, 
wherein the coefficient string dividing step is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and repeatedly perform reading of coefficients by the number 
equal to the number of coefficients in the coefficient string in the block 
of the minimum size to obtain a divided coefficient string, thereby 
obtaining the divided coefficient strings. 

[Claim 4] An image decoding method of decoding encoded 
data encoded by an image encoding method of dividing image signals 
into blocks, performing an orthogonal transform of each block, reading 
resultant orthogonal transform coefficients to obtain a coefficient string, 
and performing entropy coding thereof, the image decoding method 
comprising: 

a block selecting step of selecting a size of a block for the 
orthogonal transform, out of a plurality of blocks of different sizes; 

a decoding step for performing decoding of the encoded data by 
entropy coding adapted to a coefficient string in a block of a minimum 
size out of the plurality of blocks; and 

a coefficient string constructing step of, when a block of a size 
larger than the minimum size is selected in the block selecting step, 
constructing a coefficient string of the block of the larger size from a 
plurality of coefficient strings decoded in the decoding step. 

[Claim 5] The image decoding method according to Claim 4, 
wherein the coefficient string constructing step is configured to read 
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coefficients in the plurality of coefficient strings decoded in the 
decoding step, from the lowest in a low frequency region and write the 
coefficients read out of the respective coefficient strings, one by one in 
order into a new coefficient string from the low frequency region, 
thereby obtaining the constructed coefficient string. 

[Claim 6] The image decoding method according to Claim 4, 
wherein the coefficient string constructing step is configured to read 
coefficients in the plurality of coefficient strings decoded in the 
decoding step, from the lowest in a low frequency region and write the 
read coefficients in units of the original coefficient strings into a new 
coefficient string from the low frequency region, thereby obtaining the 
constructed coefficient string. 

[Claim 7] An image encoding apparatus of dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding thereof, the image 
encoding apparatus comprising: 

block selecting means for selecting a size of a block for the 
orthogonal transform, out of a plurality of blocks of different sizes; 

coefficient string dividing means for, when a block of a size 
larger than a minimum size is selected by the block selecting means, 
dividing a coefficient string in said block into a plurality of coefficient 
strings of a length equal to that of a coefficient string in a block of the 
minimum size; and 

encoding means for performing entropy coding adapted to the 
coefficient string in the block of the minimum size. 
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[Claim 8] The image encoding apparatus according to Claim 
7, wherein the coefficient string dividing means is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and assign the read coefficients one by one in order to the 
plurality of coefficient strings of the length equal to that of the 
coefficient string in the block of the minimum size, thereby obtaining 
the divided coefficient strings, 

[Claim 9] The image encoding apparatus according to Claim 
7, wherein the coefficient string dividing means is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and repeatedly perform reading of coefficients by the number 
equal to the number of coefficients in the coefficient string in the block 
of the minimum size to obtain a divided coefficient string, thereby 
obtaining the divided coefficient strings. 

[Claim 10] An image decoding apparatus of decoding 
encoded data encoded by an image encoding method of dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding thereof, the image 
decoding apparatus comprising: 

block selecting means for selecting a size of a block for the 
orthogonal transform, out of a plurality of blocks of different sizes; 

decoding means for performing decoding of the encoded data 
by entropy coding adapted to a coefficient string in a block of a 
minimum size out of the plurality of blocks; and 

coefficient string constructing means for, when a block of a size 
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larger than the minimum size is selected by the block selecting means, 
constructing a coefficient string of the block of the larger size from a 
plurality of coefficient strings decoded by the decoding means. 

[Claim 11] The image decoding apparatus according to Claim 
10, wherein the coefficient string constructing means is configured to 
read coefficients in the plurality of coefficient strings decoded by the 
decoding means, from the lowest in a low frequency region and write 
the coefficients read out of the respective coefficient strings, one by 
one in order into a new coefficient string from the low frequency 
region, thereby obtaining the constructed coefficient string. 

[Claim 12] The image decoding apparatus according to Claim 
10, wherein the coefficient string constructing means is configured to 
read coefficients in the plurality of coefficient strings decoded by the 
decoding means, from the lowest in a low frequency region and write 
the read coefficients in units of the original coefficient strings into a 
new coefficient string from the low frequency region, thereby obtaining 
the constructed coefficient string. 

[Claim 13] An image processing system comprising: the 
image encoding apparatus according to any one of Claims 7 to 9, and 
the image decoding apparatus according to any one of Claims 10 to 12. 

[Claim 14] An image encoding program for dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding thereof, the image 
encoding program letting a computer execute: 

a block selecting step of selecting a size of a block for the 
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orthogonal transform, out of a plurality of blocks of different sizes; 

a coefficient string dividing step of, when a block of a size 
larger than a minimum size is selected in the block selecting step, 
dividing a coefficient string in said block into a plurality of coefficient 
strings of a length equal to that of a coefficient string in a block of the 
minimum size; and 

an encoding step of performing entropy coding adapted to the 
coefficient string in the block of the minimum size. 

[Claim 15] The image encoding program according to Claim 
14, wherein the coefficient string dividing step is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and assign the read coefficients one by one in order to the 
plurality of coefficient strings of the length equal to that of the 
coefficient string in the block of the minimum size, thereby obtaining 
the divided coefficient strings. 

[Claim 16] The image encoding program according to Claim 
14, wherein the coefficient string dividing step is configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and repeatedly perform reading of coefficients by the number 
equal to the number of coefficients in the coefficient string in the block 
of the minimum size to obtain a divided coefficient string, thereby 
obtaining the divided coefficient strings. 

[Claim 17] An image decoding program for decoding 
encoded data encoded by an image encoding method of dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
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coefficient string, and performing entropy coding thereof, the image 
decoding program letting a computer execute: 

a block selecting step of selecting a size of a block for the 
orthogonal transform, out of a plurality of blocks of different sizes; 

a decoding step for performing decoding of the encoded data by 
entropy coding adapted to a coefficient string in a block of a minimum 
size out of the plurality of blocks; and 

a coefficient string constructing step of, when a block of a size 
larger than the minimum size is selected in the block selecting step, 
constructing a coefficient string of the block of the larger size from a 
plurality of coefficient strings decoded in the decoding step. 

[Claim 18] The image decoding program according to Claim 
17, wherein the coefficient string constructing step is configured to 
read coefficients in the plurality of coefficient strings decoded in the 
decoding step, from the lowest in a low frequency region and write the 
coefficients read out of the respective coefficient strings, one by one in 
order into a new coefficient string from the low frequency region, 
thereby obtaining the constructed coefficient string. 

[Claim 19] The image decoding program according to Claim 
17, wherein the coefficient string constructing step is configured to 
read coefficients in the plurality of coefficient strings decoded in the 
decoding step, from the lowest in a low frequency region and write the 
read coefficients in units of the original coefficient strings into a new 
coefficient string from the low frequency region, thereby obtaining the 
constructed coefficient string. 
[Detailed Description of the Invention] 
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[0001] 

[Technical Field to which the Invention Pertains] 

The present invention relates to an image encoding method, an 
image decoding method, an image encoding apparatus, an image 
decoding apparatus, an image processing system, an image encoding 
program, and an image decoding program capable of implementing 
efficient entropy coding of orthogonal transform coefficients in an 
orthogonal transform permitting selection among multiple block sizes. 

[0002] 

[Prior Art] 

Encoding techniques of image signals are used for transmission 
and for accumulation and reproduction of image signals of still images, 
moving images, and so on. Such techniques include known 
international standard encoding methods, e.g., ISO/IEC International 
Standard 10918 (hereinafter referred to as JPEG) as an encoding 
technique for still images, and ISO/IEC international Standard 14496-2 
(MPEG-4 Visual, which will be referred to hereinafter as MPEG-4) as 
an encoding technique for moving images. A newer known encoding 
method is ITU-T Recommendation H.264; ISO/IEC International 
Standard 14496-10 (Joint Final Committee Draft of Joint Video 
Specification, ftp//ftp.imtc-files.org/jvt-experts/2002 07 

Klagenfurt/JVT-D 157.zip, which will be referred to hereinafter as 
H.26L), which is a video coding method intended for joint international 
standardization by ITU-T and ISO/IEC. The general encoding 
techniques used in these image encoding methods are described, for 
example, in Non-Patent Document L 
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[0003] 

Image signals demonstrate close correlations between spatially 
neighboring pixels and thus transformation into the frequency domain 
leads to deviation of information to the low frequency region, which 
enables reduction of redundancy by making use of the deviation. 
Therefore, the typical image encoding methods adopt a technique of 
subjecting image signals to an orthogonal transform to transform them 
into orthogonal transform coefficients in the frequency domain, so as to 
achieve deviation of signal components to the low frequency region. 
Furthermore, the coefficient values are quantized so that small-valued 
coefficients are converted into zeros. A coefficient string is made by 
reading the coefficients in order from the lowest in the low frequency 
region and is subjected to entropy coding taking advantage of the 
deviation of coefficient values, thus achieving efficient encoding with 
reduction of redundancy. 
[0004] 

In this case, the Discrete Cosine Transform (DCT) is commonly 
used as the orthogonal transform in terms of encoding efficiency and 
ease of implementation. The orthogonal transform such as the DCT is 
carried out in units of blocks resulting from division of image signals 
into blocks each consisting of a plurality of pixels. The size of the 
blocks, as well as the property of the image signals, largely affects the 
encoding efficiency. 

[0005] 

When image signals demonstrate only small change in the 
spatial property, image signals to be transformed into orthogonal 



2002-295429 



transform coefficients in a narrow frequency region are widely 
distributed on an image, and the redundancy can be reduced more with 
increase in the size of the blocks, i.e., the size of the orthogonal 
transform, so as to increase the encoding efficiency, as compared with 
cases using smaller blocks, which raise the need for repeatedly 
expressing identical orthogonal transform coefficients. When image 
signals demonstrate large change in the spatial property on the other 
hand, the increase in the size of blocks results in obtaining various 
frequency components of orthogonal transform coefficients thereof and 
thus decreasing the deviation of coefficients, which makes efficient 
entropy coding difficult and thus decreases the encoding efficiency. 
[0006] 

In order to take advantage of the change of encoding efficiency 
due to the changes in the sizes of the blocks for the orthogonal 
transform and the property of image signals, the technology utilized is 
one of preparing orthogonal transform means in a plurality of block 
sizes in advance and adaptively selecting and using a size achieving the 
best encoding efficiency out of them. This technology is called 
Adaptive Block size Transforms (ABT) and is adopted in H.26L. Fig. 
1 shows orthogonal transform blocks used for the ABT in H.26L. The 
ABT permits a size achieving the best encoding efficiency to be 
selected out of four types of orthogonal transform block sizes shown in 
Figs. l(b)-(e), for each macroblock of 16x16 pixels shown in Fig. 1(a). 
Pixel values of each macroblock are equally divided in units of blocks 
of the selected size and are then subjected to the orthogonal transform. 
By implementing such selection, it becomes feasible to achieve 
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efficient reduction of redundancy through the use of the orthogonal 
transform in accordance with the change in the spatial property of 
image signals in the macroblocks. Reference should be made to 
H.26L as to more specific details of the ABT. 
[0007] 

The entropy coding for the orthogonal transform coefficients 
obtained by the orthogonal transform is effected on a coefficient string 
obtained by sequentially reading the orthogonal transform coefficients 
from the lowest in the low frequency region. Fig. 2(a) shows an order 
of reading coefficients in an orthogonal transform block of 4x4 pixels. 
Since the coefficients obtained by the orthogonal transform are 
arranged with the lowest frequency component (i.e., the dc component) 
at the left upper comer, the coefficients are read out in order from the 
left upper coefficient to obtain a coefficient string consisting of sixteen 
coefficients as shown in Fig. 2(b). This reading order is called 
zig-zag scan. 

[0008] 

The coefficients obtained by the orthogonal transform are 
noncorrelated with each other, and the signal components deviate to the 
low frequency region. For this reason, when they are further 
quantized, the lower frequency coefficients are more likely to be 
nonzero coefficient values, so that many zero-valued coefficients 
appear in the coefficient string. For example, it produces a sequence 
of coefficient values as shown in Fig. 2(c). Therefore, for efficient 
entropy coding of the coefficient string of this distribution, it is 
common practice in encoding of images to perform the encoding by 
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expressing the coefficient string by the numbers of continuous zero 
coefficients preceding a nonzero coefficient (runs) and coefficient 
values (levels) of the nonzero coefficients. Such encoding with runs 
and levels is also used in the entropy coding of orthogonal transform 
coefficients by the ABT. 
[0009] 

On the other hand, in order to increase the efficiency more in 
the entropy coding as described above, H.26L employs the technology 
called Context-based Adaptive Variable Length Code (CAVLC), which 
is applied to the orthogonal transform without the use of the ABT, i.e., 
to cases where the orthogonal transform is always carried out in units 
of orthogonal transform blocks of 4x4 pixels. 

[0010] 

The CAVLC in H.26L utilizes the following features: the 
maximum number of coefficients in the coefficient string obtained 
from each orthogonal transform block of 4x4 pixels is 16, the 
magnitude of runs is restricted by this maximum number, and the 
magnitude of levels tends to be larger at lower frequencies. A number 
of encoding tables used in variable length encoding are prepared as 
optimized tables for respective conditions, and they are applied while 
sequentially being switched, so as to increase the encoding efficiency. 

[0011] 

For example, in the case where runs are encoded in order, the 
first run can take a variety of values from 0 to 14 (according to the 
definition of runs in H.26L, the maximum value of runs is 14, which is 
two smaller than the total number of coefficients). On the other hand, 
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a run appearing in the last stage of the sequential encoding of runs can 
take only one of limited run values, because there is the upper limit to 
the number of coefficients in the coefficient string. Accordingly, as 
shown in Fig. 3, the right-side encoding table with the largest number 
of elements in the encoding table is applied to runs appearing in the 
initial stage, and the left-side encoding tables with the smaller number 
of elements in each encoding table are applied to runs appearing in the 
last stage. This permits assignment of codes of smaller bit counts and 
thus implements efficient entropy coding. The CAVLC achieves the 
efficient encoding by making use of the conditions such as the 
maximum number of coefficients in each block and placing restrictions 
on the range where values to be encoded can take. Reference should 
be made to H.26L as to more specific details of the CAVLC. 
[0012] 

[Non-Patent Document 1] "Basic Technologies on International 
Image Coding Standards", co-authored by Fumitaka Ono and Hiroshi 
Watanabe, CORONA PUBLISHING CO., LTD. 

[0013] 

[Problem to be Solved by the Invention] 

By applying the foregoing CAVLC to the ABT, it can be 
expected that more efficient entropy coding will also be achieved with 
the coefficient strings of the ABT. 

[0014] 

However, the CAVLC achieves the increase of encoding 
efficiency by optimizing the encoding tables in variable length coding 
for the respective conditions, based on the maximum number of 
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coefficients in blocks, and applying the encoding tables to the encoding 
with switching among them. 
[0015] 

In use of the ABT, the number of coefficients in each block 
differs depending upon blocks of different sizes; 64 in the case of 8x8 
blocks in Fig. 1(b), 32 in the cases of 8x4 and 4x8 blocks in Figs. 1(c) 
and (d), and 16 in the case of 4x4 blocks in Fig. 1(e). For this reason, 
the application of the CAVLC requires consideration to the huge 
number of conditions that can occur in the respective cases. 

[0016] 

For example, supposing the encoding tables are set according to 
the maximum number of coefficients in coefficient strings, like the 
encoding tables of runs shown in Fig. 3, the huge number of encoding 
tables must be prepared; in the case of 8x8 blocks with the number of 
coefficients being 64, it is necessary to prepare the encoding tables 
ranging from the encoding table of the number of elements being 2 to 
the encoding table of the number of elements being 62. Likewise, in 
the cases of 8x4 and 4x8 blocks with the number of coefficients being 
32, the encoding tables must be prepared from that of the number of 
elements being 2 to that of the number of elements being 30. 

[0017] 

When the entropy coding adapted to the characteristics of 
coefficients like the CAVLC was attempted to be applied to the 
orthogonal transforms selectively using the orthogonal transform 
blocks of different sizes like the ABT as described above, there was the 
problem that the number of encoding tables to be prepared became 
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huge and the memory capacity necessary for retention of the encoding 
tables became so large. Since it also involved the use of the different 
encoding tables for blocks of respective sizes and the different 
selection procedures thereof, there was the problem that the procedure 
in the entropy coding became complicated and thus implementing 
means and instrumental structure became complicated. 
[0018] 

The present invention has been accomplished in order to solve 
the above problems, and an object of the invention is to provide an 
image encoding method, an image decoding method, an image 
encoding apparatus, an image decoding apparatus, an image processing 
system, an image encoding program, and an image decoding program 
enabling efficient entropy coding in the orthogonal transform of 
variable sizes. 

[0019] 

[Means for Solving the Problem] 

In order to achieve the above object, an image encoding method 
(apparatus) according to the present invention is an image encoding 
method (apparatus) of dividing image signals into blocks, performing 
an orthogonal transform of each block, reading resultant orthogonal 
transform coefficients to obtain a coefficient string, and performing 
entropy coding thereof, the image encoding method (apparatus) 
comprising: a block selecting step (means) of selecting a size of a block 
for the orthogonal transform, out of a plurality of blocks of different 
sizes; a coefficient string dividing step (means) of, when a block of a 
size larger than a minimum size is selected in the block selecting step 
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(means), dividing a coefficient string in the block into a plurality of 
coefficient strings of a length equal to that of a coefficient string in a 
block of the minimum size; and an encoding step (means) of 
performing entropy coding adapted to the coefficient string in the block 
of the minimum size. An image encoding program according to the 
present invention is configured to let a computer execute each of the 
above steps. 

[0020] 

In the image encoding method according to the present 
invention, as described above, when a block of a large size is selected 
to be subjected to the orthogonal transform, the coefficient string in 
that block is first divided into coefficient strings and then the entropy 
coding is carried out for each of them. This permits the entropy 
coding adapted to the coefficient string in the block of the minimum 
size to be applied to the entropy coding of the coefficient string in the 
selected block, whereby it is feasible to implement efficient entropy 
coding of orthogonal transform coefficients, without complicating the 
procedure of entropy coding. 

[0021] 

In the above image encoding method (apparatus), the 
coefficient string dividing step (means) may be configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and assign the read coefficients one by one in order to the 
plurality of coefficient strings of the length equal to that of the 
coefficient string in the block of the minimum size, thereby obtaining 
the divided coefficient strings. In the above image encoding program, 
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the coefficient string dividing step executed by the computer may also 
be configured to read coefficients of the coefficient string from the 
lowest in a low frequency region and assign the read coefficients one 
by one in order to the plurality of coefficient strings of the length equal 
to that of the coefficient string in the block of the minimum size, 
thereby obtaining the divided coefficient strings. 
[0022] 

In the above image encoding method (apparatus), the 
coefficient string dividing step (means) may be configured to read 
coefficients of the coefficient string from the lowest in a low frequency 
region and repeatedly perform reading of coefficients by the number 
equal to the number of coefficients in the coefficient string in the block 
of the minimum size to obtain a divided coefficient string, thereby 
obtaining the divided coefficient strings. In the above image encoding 
program, the coefficient string dividing step executed by the computer 
may also be configured to read coefficients of the coefficient string 
from the lowest in a low frequency region and repeatedly perform 
reading of coefficients by the number equal to the number of 
coefficients in the coefficient string in the block of the minimum size to 
obtain a divided coefficient string, thereby obtaining the divided 
coefficient strings. 
[0023] 

An image decoding method (apparatus) according to the present 
invention is an image decoding method (apparatus) of decoding 
encoded data encoded by an image encoding method of dividing image 
signals into blocks, performing an orthogonal transform of each block, 
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reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding thereof, the image 
decoding method (apparatus) comprising: a block selecting step 
(means) of selecting a size of a block for the orthogonal transform, out 
of a plurality of blocks of different sizes; a decoding step (means) for 
performing decoding of the encoded data by entropy coding adapted to 
a coefficient string in a block of a minimum size out of the plurality of 
blocks; and a coefficient string constructing step (means) of, when a 
block of a size larger than the minimum size is selected in the block 
selecting step (means), constructing a coefficient string of the block of 
the larger size from a plurality of coefficient strings decoded in the 
decoding step (means). An image decoding program according to the 
present invention is configured to let a computer execute each of the 
above steps. 

[0024] 

In the image decoding method according to the present 
invention, as described above, when a block of a larger size is selected 
to implement decoding of the encoded data subjected to the orthogonal 
transform, the coefficient string in that block is constructed from 
coefficient strings of blocks included in that block. This permits a 
coefficient string to be decoded from the encoded data by the entropy 
coding adapted to the coefficient string in the block of the minimum 
size, whereby it is feasible to implement efficient entropy coding of 
orthogonal transform coefficients, without complicating the procedure 
of decoding of entropy coding. 

[0025] 
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In the above image decoding method (apparatus), the 
coefficient string constructing step (means) may be configured to read 
coefficients in the plurality of coefficient strings decoded in the 
decoding step (means), from the lowest in a low frequency region and 
write the coefficients read out of the respective coefficient strings, one 
by one in order into a new coefficient string from the low frequency 
region, thereby obtaining the constructed coefficient string. In the 
above decoding program, the coefficient string constructing step 
executed by the computer may also be configured to read coefficients 
in the plurality of coefficient strings decoded in the decoding step, from 
the lowest in a low frequency region and write the coefficients read out 
of the respective coefficient strings, one by one in order into a new 
coefficient string from the low frequency region, thereby obtaining the 
constructed coefficient string. 

[0026] 

In the above image decoding method (apparatus), the 
coefficient string constructing step (means) may be configured to read 
coefficients in the plurality of coefficient strings decoded in the 
decoding step (means), from the lowest in a low frequency region and 
write the read coefficients in units of the original coefficient strings 
into a new coefficient string from the low frequency region, thereby 
obtaining the constructed coefficient string. In the above image 
decoding program, the coefficient string constructing step executed by 
the computer may also be configured to read coefficients in the 
plurality of coefficient strings decoded in the decoding step, from the 
lowest in a low frequency region and write the read coefficients in units 



19 



2002-295429 



of the original coefficient strings into a new coefficient string from the 
low frequency region, thereby obtaining the constructed coefficient 
string. 

[0027] 

An image processing system according to the present invention 
comprises the above image encoding apparatus and the above image 
decoding apparatus. 

[0028] 

The image processing system enables efficient entropy coding 
in the orthogonal transform of variable sizes because of the provision 
of the above image encoding apparatus and also enables decoding of 
codes entropy-encoded by the above image encoding apparatus, 
because of the provision of the above image decoding apparatus. 

[0029] 

[Embodiments of the Invention] 

The preferred embodiments of the image encoding method, 
image decoding method, image encoding apparatus, image decoding 
apparatus, and image processing system according to the present 
invention will be described below in detail with reference to the 
drawings. The same elements will be denoted by the same reference 
symbols throughout the description of the drawings, without redundant 
description thereof. 

[0030] 

The description will be based on the premise that the encoding 
and decoding in the description hereinafter are implemented on the 
basis of H.26L, and the part without specific description about the 
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operation in the image encoding is supposed to conform to the 
operation in H.26L. It is, however, noted that the present invention is 
not limited to H.26L. 
[0031] 

An embodiment of the present invention will be described. In 
the encoding according to the present embodiment, concerning the 
orthogonal transform coefficients in blocks of the respective sizes in 
the ABT of H.26L, a coefficient string obtained from each block is 
divided into a plurality of coefficient strings each consisting of. 
coefficients in the number equal to the number of coefficients in a 
coefficient string of a 4x4 block. This makes it feasible to perform 
the entropy coding by the CAVLC of H.26L defined so as to be adapted 
to the 4x4 blocks. 

[0032] 

It is assumed that in the encoding, first, the ABT in H.26L is 
applied to one macroblock, a size achieving the best encoding 
efficiency is selected out of the blocks shown in Figs. l(b)-(e), and the 
orthogonal transform is effected in units of the blocks of the selected 
size. 

[0033] 

It is also assumed that the CAVLC in H.26L is employed as the 
entropy coding of orthogonal transform coefficients. Namely, it is 
assumed that only the variable length encoding adapted to the encoding 
of orthogonal transform coefficients for the 4x4 block shown in Fig. 
1(e) is defined. 

[0034] 
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For example, let us suppose herein that the 8x8 block in Fig, 
1(b) is selected. The following readout operation of reading the 
orthogonal transform coefficients is carried out for this 8x8 block. 
First, sixty four coefficients in the 8x8 block are read out by zig-zag 
scan as shown in Fig. 4(a), to obtain a coefficient string as shown in 
Fig. 4(b). 

[0035] 

Then this coefficient string is divided into four coefficient 
strings each consisting of sixteen coefficients, the number of which is 
the same as the number of coefficients in the coefficient string of the 
4x4 block. Here the coefficients in the original coefficient string are 
read out from the lowest in the low frequency region and alternately 
assigned to the four coefficient strings, thereby obtaining the 
coefficient strings after division. Figs. 4(c) and (d) show this readout 
operation. Since the coefficients are alternately assigned to the 
respective coefficient strings from the lowest in the low frequency 
region, the first divided coefficient string is assigned the coefficients 
read out in the order of the zeroth, fourth, eighth, twelfth,... in the 
original coefficient string, and the second divided coefficient string is 
assigned the coefficients read out in the order of the first, fifth, ninth, 
thirteenth,... in the original coefficient string. The third and fourth 
divided coefficient strings are not illustrated in Fig. 4. 
[0036] 

Similarly, when the 8x4 block or 4x8 block of Fig. 1(c) or (d) is 
selected, thirty two coefficients are divided into two coefficient strings 
each consisting of sixteen coefficients. The readout method for 
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obtaining the divided coefficient strings is also similar to that in the 
case of the 8x8 block except that the number of coefficient strings 
alternately assigned the coefficients is 2, instead of 4; the coefficients 
in the original coefficient string are read out from the lowest in the low 
frequency region and alternately assigned to the two coefficient strings. 
[0037] 

The coefficient strings obtained in this way are entropy encoded 
according to the same procedure as the encoding of CAVLC without 
the use of the ABT, and the encoded data is outputted in order as 
encoded data of orthogonal transform coefficients in the ABT block. 

[0038] 

At this time, the CAVLC of H.26L utilizes the space context to 
switch the applied encoding table on the basis of the number of 
nonzero coefficients in an adjacent 4x4 block. For this reason, 
arrangements of coefficient strings after division in the ABT blocks are 
defined for the ABT blocks larger than the 4x4 block. The definitions 
are presented in Fig. 5. For example, the 8x8 block shown in Fig. 
5(a) is handled on the assumption that the first divided coefficient 
string illustrated in Fig. 4(c) is located at the position of "1" and the 
second divided coefficient string illustrated in Fig. 4(d) at the position 
of "2." It is assumed that, using the definitions of the arrangement, 
the space context for the divided coefficient strings in the ABT blocks, 
or the space context for 4x4 blocks adjacent to the ABT blocks is 
handled in the same manner without any change as in the technique in 
the CAVLC of H.26L. 
[0039] 
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In the decoding, the original orthogonal transform matrix can 
be obtained according to the procedure reverse to the procedure in the 
encoding, 

[0040] 

Let us suppose that the ABT in H.26L is applied to one 
macroblock, a size is designated out of the blocks shown in Figs. 1(b) 
to (e), and encoded data from this macroblock is one resulting from the 
orthogonal transform carried out in units of ABT blocks of the size. 

[0041] 

At this time the encoded data contains encoded data obtained 
by entropy coding of the divided coefficient strings by the CAVLC, in 
order as encoded data of orthogonal transform coefficients in the ABT 
blocks. Accordingly, it is sequentially decoded according to the 
procedure of CAVLC to obtain the coefficient strings after the division. 

[0042] 

Since these divided coefficient strings are coefficient strings 
divided by the readout method shown in Fig. 4, the original orthogonal 
transform coefficient block can be obtained by conversely writing the 
coefficients of the divided coefficient strings into each original 
coefficient string and further writing the resultant coefficient strings 
into the orthogonal transform coefficient block. The procedure 
thereafter is the same as the decoding procedure with application of the 
ABT in H.26L. 

[0043] 

A configuration of image processing system 1 for implementing 
the above image encoding and image decoding will be described below. 



2002-295429 



Fig. 7 is a block diagram showing image processing system 1 
according to an embodiment. The image processing system 1 is 
composed of image encoding apparatus 10 for encoding image data, 
and image decoding apparatus 20. 
[0044] 

The image encoding apparatus 10 has orthogonal transform unit 
11, coefficient string divider 12, entropy encoder 13, and block selector 
1 4. The orthogonal transform unit 1 1 has a function of performing the 
orthogonal transform of image data to transform it into frequency 
components. The orthogonal transform unit 11 divides each 
macroblock of the image data into a plurality of blocks and performs 
the orthogonal transform of the divided blocks. It is connected to 
block selector 14, and the block selector 14 selects a block capable of 
implementing the orthogonal transform with the best efficiency. 
[0045] 

The coefficient string divider 12 has a function of dividing a 
coefficient string obtained by the orthogonal transform, into 
coefficient strings of a predetermined length. Here the term 
"predetermined length" is defined as a length of a coefficient string 
obtained by the orthogonal transform of a block of a minimum size 
out of the blocks into which a macroblock can be divided for the 
orthogonal transform of image data by orthogonal transform unit 11. 
[0046] 

The entropy encoder 13 has a function of encoding the 
coefficient strings divided by the coefficient string divider 12. Since 
the coefficient string divider 12 divides the original coefficient string 
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into the coefficient strings of the same length as that of the coefficient 
string of the minimum block as described above, the entropy encoder 
13 can be configured to be adapted for encoding of the coefficient 
strings of that length and can perform efficient encoding. 
[0047] 

The image decoding apparatus 20 has entropy decoder 23, 
coefficient string constructor 22, inverse orthogonal transform unit 21, 
and block selector 24. The entropy decoder 23 has a function of 
decoding encoded data. 

[0048] 

The coefficient string constructor 22 has a function of 
constructing an original coefficient string from coefficient strings 
divided upon encoding by image encoding apparatus 10. The 
coefficient string constructor 22 is connected to block selector 24, 
acquires information about the size of the original block from the block 
selector 24, and constructs the original coefficient string on the basis of 
the information. The block selector 24 is able to acquire the size of 
the original block, based on additional information or the like 
transmitted with the encoded data from the image encoding apparatus. 

[0049] 

The inverse orthogonal transform unit 21 has a function of 
transforming the coefficient strings constructed by the coefficient string 
constructor 22, into image data. 

[0050] 

The image processing system 1 of the structure as described 
above implements the aforementioned image encoding and image 
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decoding, 

[0051] 

In the present embodiment the zig-zag scan was applied to 
readout of orthogonal transform coefficients, but the readout method of 
coefficients in application of the present invention does not have to be 
limited to the zig-zag scan. For example, the present invention may 
also be applied to cases of application of the field scan for field 
encoding in interlaced images, which is defined in the ABT of H.26L. 
In this application, the dividing technique of the coefficient strings in 
the present embodiment can also be applied as it is. 

[0052] 

The present embodiment showed the alternate readout method 
as shown in Fig. 4, as a readout method for obtaining the coefficient 
strings after the division, but it is also possible to obtain the coefficient 
strings after the division by another readout method different therefrom. 
For example, as shown in Figs. 6(c) and (d), sixteen consecutive 
coefficients each are read out from the original coefficient string from 
the lowest in the low frequency region and each series is assigned to 
one of the coefficient strings after the division. 

[0053] 

In the present embodiment, the readout of orthogonal transform 
coefficients in the encoding is implemented so as to first perform the 
first readout for obtaining the coefficient string from the orthogonal 
transform block and then perform the second readout for obtaining a 
plurality of coefficient strings after the division. The writing of 
orthogonal transform coefficients in the decoding is implemented so as 



2002-295429 



to first perform the first writing for obtaining the coefficient string after 
construction and then perform the second writing for obtaining the 
orthogonal transform block. However, the readout and writing of 
coefficients according to the present invention do not have to be limited 
to these methods, but may also be implemented by a variety of readout 
and writing methods that can obtain coefficient strings in the desired 
arrangement. For example, it is also possible to implement such 
readout as to immediately obtain a plurality of divided coefficient 
strings in the first coefficient readout from the orthogonal transform 
block. The writing from the divided coefficient strings may also be 
modified so as to immediately obtain the orthogonal transform block in 
the first coefficient writing. 
[0054] 

In the present embodiment, the coefficient strings after the 
division were arranged as shown in Fig. 5, and the space context from 
the adjacent 4x4 block in the CAVLC in H.26L was assumed to be 
handled without any change. In this regard, it can also be 
contemplated that the coefficient strings divided from the coefficient 
strings of the ABT blocks larger than the 4x4 block are originally 
different in the property from the coefficient strings in the case of the 
4x4 block and weights are given to numerals used as the space context. 
Specifically, where the number of nonzero coefficients is used as the 
space context from the adjacent block, a constant is always added to or 
multiplied by the number of nonzero coefficients in each divided 
coefficient string obtained from an ABT block larger than the 4x4 
block, when used as the space context. In another configuration, 
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where the coefficient strings after the division are obtained by 
continuously reading out the coefficients from the lowest in the low 
frequency region as shown in Fig, 6, different constants may be added 
to or multiplied by coefficients read out from the low frequency region 
and coefficients read out from the high frequency region, 
[0055] 

The embodiments were described on the premise that the 
encoding and decoding were substantialized on the basis of H.26L and 
were described based on the ABT and CAVLC in H.26L. However, it 
is noted that the image encoding methods to which the present 
invention can be applied are not limited to H.26L, and the present 
invention can be applied to various image encoding methods permitting 
selection among a plurality of sizes of blocks for the orthogonal 
transform and using the entropy coding adapted to the orthogonal 
transform coefficients. 

[0056] 

[Effects of the Invention] 

The image encoding method, image decoding method, image 
encoding apparatus, image decoding apparatus, and image processing 
system according to the present invention provide the following effect, 
as detailed above. Namely, where a size of a block for the orthogonal 
transform can be selected from a plurality of sizes, a coefficient string 
consisting of resultant orthogonal transform coefficients is divided into 
a plurality of coefficient strings of the same size as that of the 
coefficient string in the block of the minimum size and each of these 
coefficient strings is subjected to the entropy coding adapted to the 
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coefficient string in the block of the minimum size; whereby it is 
feasible to implement the efficient entropy coding, without increase in 
the number of encoding tables in the entropy coding and without 
complicating the encoding tables and the procedure of selection 
thereof. 

[Brief Description of the Drawings] 
[Fig- 1] 

A figure including diagrams showing the orthogonal transform 
blocks used in the Adaptive Block size Transforms (ABT) of H.26L. 
[Fig. 2] 

A figure including diagrams showing a readout method of 
reading coefficients in a 4x4 block, and an example of a coefficient 
string after readout. 

[Fig. 3] 

A diagram showing the encoding tables of runs used in the 
Context-based Adaptive Variable Length Code (CAVLC) of H.26L. 
[Fig. 4] 

A figure including diagrams showing an example in which a 
method of readout and division of orthogonal transform coefficients 
according to the present invention is carried out for an 8x8 block. 

[Fig. 5] 

A figure including diagrams showing definitions of 
arrangement in original blocks, of coefficient strings after division 
according to the present invention. 

[Fig. 6] 

A figure including diagrams showing an example in which 
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another method of readout and division of orthogonal transform 
coefficients according to the present invention is carried out for an 8x8 
block. 

[Fig. 7] 

5 A block diagram showing a configuration of an image 

processing system according to an embodiment. 
[Explanation of Reference Numerals] 

1 - image processing system, 10 - image encoding apparatus, 11 

- orthogonal transform unit, 12 - coefficient string divider, 13 - entropy 
10 encoder, 14 - block selector, 20 - image decoding apparatus, 21 - 

inverse orthogonal transform unit, 22 - coefficient string constructor, 23 

- entropy decoder, 24 - block selector. 
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[Document Name] Abstract 
[Abstract] 

[Object] To provide an image encoding apparatus capable of achieving 
efficient entropy coding in the orthogonal transform of variable sizes, 
[Means of Solution] Image encoding apparatus 10 according to the 
present invention is an image encoding apparatus 10 for dividing image 
signals into blocks, performing an orthogonal transform of each block, 
reading resultant orthogonal transform coefficients to obtain a 
coefficient string, and performing entropy coding, which has block 
selector 14 for selecting a size of a block for the orthogonal transform, 
out of a plurality of blocks of different sizes; coefficient string divider 
12 for, when a block of a size larger than a minimum size is selected by 
block selector 14, dividing a coefficient string in the block into a 
plurality of coefficient strings of a length equal to that of a coefficient 
string in a block of the minimum size; and encoder 13 for performing 
entropy coding adapted to the coefficient string in the block of the 
minimum size. 
[Selected Drawing] Fig, 7 
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